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Significance for public liealtli 

Incident Reporting Systems (IRS) are and will continue to be an important 
influence on improving patient safety. However, they are not the panacea 
that many believe them to be. They have several limitations that should be 
considered when utilizing them or interpreting their output: i) IRS can't be 
used to measure safety (error rates); ii) IRS can't be used to compare organ- 
izations; iii) IRS can't be used to measure changes over time; iv) IRS gener- 
ate too many reports; v) IRS often don't generate in-depth analyses or result 
in strong interventions to reduce risk; vi) IRS are associated with costs. 
Moving forward, several strategies are suggested to maximize their value: i) 
make reporting easier; ii) make reporting meaningful to the reporter; iii) 
make the measure of success system changes, rather than events reported; 
iv) prioritize which events to report and investigate, do it well; v) convene 
with diverse stakeholders to enhance their value. 



Abstract 

Incident Reporting Systems (IRS) are and will continue to be an 
important influence on improving patient safety. They can provide 
valuable insights into how and why patients can be harmed at the 
organizational level. However, they are not the panacea that many 
believe them to be. They have several limitations that should be con- 
sidered. Most of these limitations stem from inherent biases of volun- 
tary reporting systems. These limitations include: i) IRS can't be used 
to measure safety (error rates); ii) IRS can't be used to compare 
organizations; iii) IRS can't be used to measure changes over time; iv) 
IRS generate too many reports; v) IRS often don't generate in-depth 
analyses or result in strong interventions to reduce risk; vi) IRS are 
associated with costs. IRS do offer significant value; their value is 
found in the following: i) IRS can be used to identify local system haz- 
ards; ii) IRS can be used to aggregate experiences for uncommon con- 
ditions; iii) IRS can be used to share lessons within and across organ- 
izations; iv) IRS can be used to increase patient safefy culture. Moving 
forward, several strategies are suggested to maximize their value: i) 
make reporting easier; ii) make reporting meaningful to the reporter; 
iii) make the measure of success system changes, rather than events 
reported; iv) prioritize which events to report and investigate, report 
and investigate them well; v) convene with diverse stakeholders to 
enhance the value of IRS. 



The promise 

Incident Reporting Systems (IRS) are a cornerstone for improving 
patient safety.' All high-risk industries have them. While IRS are rela- 
tively new in healthcare, similar systems in nuclear, railway, fire, and 
aviation industry have had tremendous success.-^ The concept behind 



IRS is simple; they provide a mechanism to identify risks so that 
organizations can implement interventions to reduce these risks. IRS 
provide valuable information to identify hazards and surface learning 
opportunities. In healthcare, IRS provide frontline caregivers a mech- 
anism to raise concerns, providing voice to these clinicians that man- 
agement can work to mitigate. 

The Institute of Medicine (lOM) advocates for the development and 
use of IRS. The lOM recommended the following. Recommendation 
5.1: a nationwide mandatory reporting system should be established 
that provides for the collection of standardized information by govern- 
ments about adverse events that result in death or serious harm. 
Recommendation 5.2: the development of voluntary reporting efforts 
should be encouraged.^ The Joint Commission now requires that all 
hospitals have and use IRS. 

Because of this, there has been an explosion of adverse event 
reporting systems in healthcare. Some reporting systems are national 
(Australian Incident Management System, National Reporting 
Learning System, etc.), some are local (Patient Safety Network, 
Pennsylvania Safety Reporting System, etc.), some focus on a specific 
area (Intensive Care Unit Reporting System) while others focus on a 
specific type of event (Medmarx - medication), some are public while 
others are private. The recent Patient Safety Act has tasked the gov- 
ernment with creation of a national database for sharing of adverse 
events, known as the National Patient Safety Database (NPSD). 

Yet to date, there is limited evidence to demonstrate that IRSs in 
healthcare have lived up to their expectations for making care safer. 
Progress in reducing preventable harm has been slow,'* '' and IRS have 
not provided the insight or lead to improvements that many had hoped 
for. In this article, we review some challenges that IRS have encoun- 
tered, focusing on what IRS cannot do. We then discuss what they can 
do and how they provide the most value. Finally, we offer some practi- 
cal suggestions on moving forward. 



The challenge 

Incident Reporting Systems can't be used to measure 
safety (error rates) 

IRS cannot be used to measure safefy. There is growing interest 
among policy makers, payers, hospital leaders and patients in measur- 
ing how safe hospitals and health systems are. This includes: i) meas- 
uring the rate of specific adverse events (for example, how often do 
medication errors occur in our institution?); ii) holding 
individuals/organizations accountable for safety problems (for exam- 
ple, why are never events occurring at this institution?); iii) measuring 
the overall safety profile of a hospital to make purchasing decisions 
(for example, is this a safe hospital? Should I get my care here?). 

In the absence of better sources of information, some turn to IRS to 
answer these questions. However, IRS data are a non-random sample 
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from the total universe of safety hazards. They are more suited for 
identifying risiis. For many reasons, IRS cannot be used to measure 
safety. First, events are under-reported. Approximately 7% of ail adverse 
events are reported to IRS,^ and different methods of detecting errors 
identify different types of events and results in different conclusions 
about safety.'' Reasons for this include fear of retribution, shame, lacii 
of time, complexity of reporting system, lack of perceived value, report- 
ing fatigue, etc' Moreover, there is variation (bias) on what types of 
events are reported. For example, is a tear during vaginal delivery of a 
baby an adverse event? While some s&y yes, others may argue that it is 
a known and unavoidable consequence of vaginal delivery of large 
babies. Even among reported events, there is variation in the threshold 
in reporting. While some individuals may report near-misses, others 
only report those event that result in patient harm. Some types of 
adverse events are reported with high frequency (e.g. falls) while oth- 
ers are often under-reported (e.g. medication adverse events). Finally, 
some provider types report adverse events with regularity (e.g. nurses) 
while others report events infrequently (physicians).* 

In order to be a valid measure of the rate of adverse events, a meas- 
ure requires three things. There should be a clear definition of the 
event (numerator); few adverse event in healthcare are well defined. 
There should be a clear definition of the population at risk (denomina- 
tor); the population in healthcare are usually not defined. Finally, there 
should be a consistent surveillance system for detection of both the 
event and the population at risk. IRS have a problem with all three of 
these. For example, medication adverse events might be measured in 
terms of patients, patient days, or doses of medications administered; 
IRS don't measure any of these. Moreover IRS rely on the vigilance, 
honesty, and whim of healthcare providers to detect and report adverse 
events; this is a poor substitute for a surveillance system. The most 
valid measures of patient safety are related to hospital acquired infec- 
tions, such as central-line associated blood stream infections. For many 
of these conditions, there are national definitions for numerators, 
denominators, and hospitals have invested significant infrastructure 
into active surveillance processes. Incidentally, this is the area with the 
most demonstrated progress.' 

Incident Reporting Systems can't be used to compare 
organizations 

IRS cannot be used to compare organizations. Healthcare leaders 
and patients want a measure to compare patient safety among health 
systems, hospitals, and healthcare providers. The underreporting and 
bias in the system make comparisons nearly impossible. This is under- 
scored by a statement from the National Coordinating Council for 
Medication Error Reporting Systems (NCC-MERP) explicitly recom- 
mending that data from these systems not be used to compare health- 
care providers or their organizations.'" For example. Table 1 is a hypo- 
thetical example of the number of medication error reported to a hos- 
pital IRS. On initial inspection, it may seem that the oncology unit has 
poor medication safety practices. What if the oncology unit sees four 
times more patients or administers four times more medications per 
patient than the other units? As it turns out, an investigation into these 
numbers revealed that the oncology unit was more vigilant in its med- 
ication safety practices because of its use of chemotherapeutic agents. 
There were twice as many double checks as the other units (better safe- 
ty culture). Near-misses uncovered by these double-checks were rou- 
tinely reported to the IRS (better reporting culture). Blindly using the 
IRS to compare these units would have misled us about their safety 
practices. 

Incident Reporting Systems can't be used to measure 
clianges over time 

Valid error rates are required to make inferences about changes in 



safety over time. For all of the reasons discussed above, IRS do not pro- 
vide such information. Take the example of wrong-site surgeries. The 
Veterans Affairs (VA) has developed a tool to reduce their risk, and in 
2004 The Joint Commission required that hospitals implement a time 
out, to prevent these events. Yet despite implementation of these inter- 
ventions, the apparent rate of wrong-site surgeries from IRS continues 
to climb (Figure 1). This increase is much more likely to be due to 
increased reporting (reporting bias) from increased awareness rather 
than an increase in wrong-site surgery from all these interventions. 
This data urges caution in interpreting changes over time in reported 
events, even highly visible events that are well-defined. 

Incident Reporting Systems generate too many 
reports 

Even for a medium-sized organization, an active error reporting cul- 
ture can lead to many reports. For example, Johns Hopkins Hospital 
(~1000 beds) generates approximately 500 reports per month. Many 
organizations do not have the resources to read, much less analyse all 
of these reports. This can lead to dissatisfaction from users when their 
reports are left unresolved. In addition, reporting is sometimes used to 
complain about a colleague's performance or behaviour. It is not sur- 
prising that many users feel dissatisfied with these systems." 

Moreover, some reports provide little incremental value about the 
insight of safety systems. For example, patient fall reports tend to con- 
tain the following similar narrative: nurse found patient on the ground 
after the patient tried to get out of bed. Physician was notified. After 
evaluation, no permanent harm was identified. Without further details 
and/or action by those analysing team, these reports tend to have lim- 
ited incremental value, questioning the benefit of having the user 
report them in the first place. 

Incident Reporting Systems often don't generate in- 
deptli analyses or result in strong interventions to 
reduce risk 

Because of the relatively limited resources, error investigations and 
analysis in healthcare are often superficial. In many hospitals, the legal 
department performs these analyses. Their staff often have limited to no 
training in adverse event investigation or human factors. This is con- 
trasted with other high-risk industries, where accidents are rare and 
investigations are in-depth. For example, the United Airlines flight 1549 
Hudson River crash resulted in a 16-month investigation, removal of the 
plane from the bottom of the river, and DNA analysis of the Canadian 
Geese that caused the engine failure. Examples such as this are rare in 
healthcare. Moreover meaningful change occurs infrequently, with the 
majority of changes being informing staff involved and education/train- 
ing.^'^ This leads to recurrence of adverse events across the healthcare 
industry, sometimes even within the same institution.''* The lack of 
meaningful change can be due to a variety of reasons: production pres- 
sures, small financial margins, limited regulatory authority, lack of stan- 



Table 1. Hypothetical example of number of adverse events 
reported to a hospital Incident Reporting Systems. 



1 Clinical area 


Events/month 1 


Emergency Department 


25 


General Internal Medicine 


19 


Oncology 


112 


Surgery 


15 


Intensive Care Unit 


30 
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dardization, professional culture, lack of institutional will, etc.^ '^ The 
lack of meaningful change, for whatever reason, diminishes the poten- 
tial value of an adverse event reporting system. Some challenges are 
national, leaving individual hospitals with only a limited ability to make 
changes. For example, a medication error will likely occur somewhere in 
the United States today involving an inadvertent hydromorphone for 
morphine substitution. Yet the individual hospital has a limited ability to 
fix the problem of two look-alike sound-alike drugs that have an 8X dif- 
ference in potency. Instead, many hospitals resort to education and train- 
ing of their staff, an intervention known to have limited ability to perma- 
nently reduce risk. 

Incident Reporting Systems are associated witli costs 

IRS come with costs. The costs of building and implementation of these 
systems are most visible. However, there are significant costs to training 
staff on its use, actual reporting, collecting, and analysing the data from 
these systems. These costs are often ignored, but can be significant. In 
the fall example above, the costs of collecting additional informational 
about fall events might be better spent on implementing best practices 
(fall risk assessment, fall risk communication/signs, hourly rounds for 
comfort, safety, pain, toileting, chairA)ed/posey alarms, medication modi- 
fication) to reduce patient falls. Moreover, when budgets are tight, it is 
easy to cut funds for the analysis and change implementation from these 
systems. Ironically, these are the steps that improve patient safety. On the 
other side, the assumption is that IRS will lead to reduction of medical 
errors. This in turn, would lead to reduction in patient harm, which would 



lead to reduction in costs. While true in principle, this has been extreme- 
ly difficult to quantify and demonstrate. As with most preventative efforts, 
benefits are theoretical while costs are real. For example, hospitals are 
compensated for the complexity and number of patients they treat, not the 
number of errors they prevent. In some cases, having an adverse event 
might lead to a higher profit margin. 



What can Incident Reporting Systems do 

Given these challenges, what value, can an adverse event reporting 
system offer? Should a hospital/healthcare system invest in one at all? 

Incident Reporting Systems can be used to identify 
local system hazards 

Information provided in IRS are nuggets of gold from which we may 
better identify hazards, understand the complex inner workings of our 
healthcare system and design interventions to reduce risks to future 
patients. For example, several years ago, a nurse at Johns Hopkins 
Hospital reported that her patient was sent the incorrect medication 
from pharmacy Although the label was correct, she noticed that the pill 
looked different than the metoprolol that she normally gave. As it turns 
out, the pharmacy label-maker had received a software update and had 
defaulted back to incorrect settings. Such a report, of a near-miss, like- 
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Figure 1. Wrong-patient, wrong-site, wrong-procedure events reviewed by the joint commission. Reproduced with permission. 
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ly prevented hundreds of medication errors. Currently, the most valu- 
able lessons from these IRS occur from single cases. A variety of tools 
that provide a systematic analytic framework can help clinicians learn 
from these events.'' 

Incident Reporting Systems can be used to aggregate 
experiences for uncommon conditions 

While the most valuable lessons from IRS occur from single cases, 
the aggregation of uncommon cases can provide insight into failure 
modes and risks. In the wrong-site surgery example above, aggregation 
of 132 wrong-site surgical events in the state of Colorado revealed that 
100% of the cases involved a communication failure and 72% did not 
have a time-out performed.'* This required 6 years and data from 5937 
physicians. A single institution would not have enough experience/ 
events to identify such patterns and potential vulnerabilities. 
Therefore, the aggregation and analysis of uncommon cases from the 
IRS across many organizations can be useful in this way. 

Incident Reporting Systems can be used to sliare 

lessons within and across organizations 

The lessons learned from IRS can be used to educate, inform, and 
prevent other organizations from experiencing the same adverse 
events. Such a system for sharing can occur at a local, regional, nation- 
al, or international level. For example, the Canadian Global Patient 
Safety Alerts (GPSA) system is a repository of case details and lessons 
learned from adverse events. Healthcare organizations can search the 
system to identify adverse events, their known failure mechanisms, 
and interventions that may be implemented to prevent their occur- 
rence. This allows healthcare organizations to share the most valuable 
aspect of IRS (lessons learned) without the burdens of incident inves- 
tigation and confidentialify concerns. Given that the same adverse 
events are occurring around the world, such sharing is sorely needed. 

Incident Reporting Systems can be used to increase 
patient safety culture 

The real implementation and use of a reporting system (the full 
reporting cycle with: reporting, analysis, and implementation of system 
changes in analogy to the Plan-Do-study-Act cycle of quality manage- 
ment) within a healthcare institution communicates a lot about how 
the organization views patient safety. The implementation and commu- 
nication of such a system to staff members can be a method of chang- 
ing patient safety culture. This effect is apart from the advantages of 
the IRS itself. 



Moving forward 

To enhance their value, we offer some practical suggestions in the 
design and use of IRS. 

Make reporting easier 

As opposed to making adverse event reporting systems more com- 
plex, we should move to making reporting exceedingly easy and less 
burdensome. Healthcare providers should have quick and ready access 
(electronic, web-based) to these systems.'^ These systems should be so 
simple that staff can use them with minimal or without training.^" 
Given the infrequent nature by which most staff members report 
adverse events, any training will likely have been forgotten at the time 
of reporting. 



These systems should probably ask for a minimal amount of informa- 
tion. Instead of asking the healthcare provider to categorize the event, 
rate the event, and attribute causes, a free text description and some 
identifying information might be all that is required. The deluge of 
adverse event reports by healthcare providers has lead to many reports 
that are incomplete,'^' with some being inaccurate." A patient safety 
officer familiar with adverse event reporting should further investigate 
events that have merit. Such a system has been developed at the 
University of Basel, Switzerland and has been adopted as the standard 
in Germany.^^ 

Make reporting meaningful to the reporter 

Healthcare providers do not report adverse events because much of 
the time, identified problems are not remediated.'^^ Moreover, the 
reporter rarely receives feedback about whether the event was read or 
deemed important. From a communication science perspective, this is 
problematic because communication has not taken place until the 
receiver of a message has conveyed feedback to the sender to confirm 
message receipt and validated the intended meaning of the message. 
Thus, interpersonal communication is a simultaneous sense-making 
process, and the extent to which participants share a common interpre- 
tation of a given message determines the effectiveness of any given 
interaction.'*'' In light of this literature, it is no surprise that when sur- 
veyed, less than 39% felt that reporting adverse events represented 
good use of staff time and resources." Instead, clinicians often devel- 
op apathy towards reporting, especially in the setting of high produc- 
tion pressures. 

The following communication concepts can help clarify how to make 
incident reporting more meaningful to the reporter. 

Empathy {i.e., one's abilify to understand another's thoughts and 
feelings) is a critical skill that contributes to competent communica- 
tion.'*^ This particularly makes sense in the IRS context: IRS managers 
should take the perspective of clinicians, understand why they are cur- 
rently not reporting, and then adapt their communication strategies to 
these reasons to facilitate increased reporting. 

Communication relies on multiple channels. Written communica- 
tion is channel-lean and thus subject to a high likelihood of miscom- 
munication {i.e., message being decoded inaccurately).^^ In the con- 
text of AERS, this risk goes both ways - the report of an adverse event 
may be interpreted inaccurately by risk management, and the response 
by risk management may be interpreted inaccurately by the reporting 
medical professional. All behaviour (and non-behaviour) is communi- 
cation; thus, no feedback in response to AERS communicates some- 
thing (such as, for example, they don't care to respond). 

Communication has literal meanings as well as relational implica- 
tions.^^ For example, communication in response to submitted incident 
reports contains informational content {e.g., your report was received), 
but it also contains relational content that, if done well, can convey 
trust, investment, commitment, etc.; these relational contents define 
the type of relationship we want to have with others. So conceptually, 
relational message contents define the type of communication that 
takes place in a given organization. Thus, relational communication 
chronologically precedes the development of a safety culture, so it could 
be even argued that it is communication culture which leads to safer 
practice. 

Based upon these communication concepts, the following strategies 
are recommended: 

The person who reports an event should receive timely feedback.^^" 
Even if this feedback is only to thank the reporter and let them know 
that the event is being investigated. This feedback validates under- 
standing of the report and communicates that somebody is listening 



OPEN /^ACCESS 



[Journal of Public Health Research 2013; 2:e27] 



[page 157] 



Review 



and cares about the event. Furttiermore, a competent communication 
response can influence reporters to feel tfiat reporting is worth my time 
and may increase future reporting. 

Managers should share reports with staff.^' This will not only educate 
the staff of risks in their environment, but will also provide a forum to 
solicit their ideas on how to further reduce risk and inform staff that 
action is occurring as a result of their reporting efforts. Moreover, these 
lessons should be shared at the local, regional, national, and interna- 
tional levels. This may reduce the likelihood that events recur across 
the healthcare industry. 

Leaders should devote institutional resources to not just collecting 
the data, but analysing the events and mediating risk. In the end, this 
is the ultimate value of the system. When staff observe that the institu- 
tion is willing to change based upon their feedback, real changes in 
safety culture start to occur. 

Make the measure of success system changes, rather 
than events reported 

It is easy to become focused on gathering reports from an adverse 
event reporting system. In the early stages of implementation, it is 
important to educate, inform, and get staff to report events. This is an 
important component of safety culture. However, the ultimate measure 
of success of an IRS is not the number of reports received, rather the 
amount of harm prevented as a result of the system. In many cases, this 
might be difficult to measure. A reasonable alternative is measuring 
the number of system changes made as a result of the IRS. 

Prioritize which events to report and investigate; 
report and investigate them well 

Whereas the airline industry has fatal accidents as a solitary focus, 
healthcare has no such focus. This lack of focus has caused the health- 
care industry to fragment its efforts. As mentioned above, this has lead 
to a deluge of adverse event reports with few in-depth analyses and 
even fewer strong solutions. 

One alternative is to focus on reporting of a finite set of high-yield 
events. Such a set of events might be based upon level of harm, pre- 
ventability, or regional/national priority. An organization might focus 
on events that occur most frequently (e.g., medication errors), lead to 
the most harm (e.g., falls), or are of greatest concern to patients or pol- 
icy makers (e.g., wrong-site surgery). The National Quality Forum 
(NQF) has such a set of events, termed Serious Reportable Events 
(SRE).2* Mandatory reporting of some form of SRE has been adopted by 
over 11 states.^' This will serve to prioritize healthcare's efforts and 
might lead to a more meaningful IRS. 

Convene with diverse stakeholders to enhance the 
value of Incident Reporting Systems 

Several countries have a national IRS (United Kingdom, Australia, 
United States developing). These recommendations for local organiza- 
tions may also apply to national systems. Moreover, those leading 
national IRS should meet regularly with healthcare provider organiza- 
tions and other stakeholders to improve use of IRS data to reduce pre- 
ventable harm. For example, a national reporting system may partner 
with device manufacturers or professional specialties to share common 
events. 



Conclusions 

IRS are and will continue to be an important influence on improving 
patient safety. They can provide valuable insights into how and why 



patients can be harmed at the organizational level. However, they are 
not the panacea that many believe them to be. They have several limi- 
tations that should be considered when utilizing them or interpreting 
their output. Moving forward, the suggested strategies may maximize 
their value. 
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